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Application No. 60/463,699, filed April 17, 2003; U.S. Patent Application No. 
60/515,962. 

BACKGROUND OF THE INVENTION 

A. Field of the Invention 

[0002] The present invention is directed to a product development and assessment 
system and method. Specifically the invention is directed to a method, system, and 
program product for conducting network-based surveys of respondents (such as 
prospective or existing consumers or customers) using continuous metric scales; 
developing a framework within which respondents rate new or existing products, services 
or advertisements; translation of the survey results into a readily understandable score, and 
automatic interpretation and reporting of the data and score in plain English. 

B. Background 

[0003] Traditionally, product and service development involves evaluation of 
customer acceptance. During economic slowdowns, companies typically have fewer 
resources to devote to time and cost intensive customer studies. A rapid, reliable and 
inexpensive system for getting large scale consumer evaluations of new product concepts, 
and developing these concepts into full implementation and execution would be desirable. 

[0004] Furthermore, both venture capitalists and investors capitalized Internet start- 
ups at historic levels — even though the start-ups often had little marketplace experience, 
no profits, and no measure of whether customers would adopt their product or service over 
the long run. The bottom dropped out of this market. Some of the companies were 
popular, but did not have profit models to sustain them. Other companies failed because 
customers were dissatisfied with either the product or the execution of the product, i.e., 
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how easy or charming was the experience of doing business on-line. One factor leading to 
the funding debacle was that venture capitalists did not have the tools to assess whether 
consumers would try, like, and adopt the new electronic product or service. The new 
companies focused their capital on building expensive systems, databases, 
communications systems, applications and even factories and advertising before they 
knew whether customers really wanted the product or service as the company planned to 
deliver it. 

[0005] Investors typically rely on business plans to determine whether or not to invest. 
Business plans, however, typically lack a reliable objective indicator of: whether the target 
customer wants the idea; whether, if consumers try it, it delivers what the idea promises; 
and whether the customer perceives that the electronic product or service is an effortless, 
charming experience that works at the customer's speed and is an improvement over the 
customer's old way of acquiring the product or service. 

[0006] Today, many companies use focus groups to estimate customer response to 
new concepts, products, and advertisements. Focus groups can identify issues, anecdotes 
and customer language. The results, however, are often narrow. Focus group results, 
moreover, can readily be manipulated by the subjective nature of the questions and the 
moderator's position on the product, service, or advertisement. Focus groups have their 
use, but focus groups cannot deliver rigorous quantitative and qualitative evaluation and 
diagnosis of the concept, product, and advertisement. 

[0007] Those skilled in the art appreciate that the most reliable indicator of the 
potential acceptance of a product is "rank order data." Rank order data is a measurement 
of the customer's view on the relative appeal of the new product concept compared to the 
customer's current product. Unfortunately, obtaining valid and reliable rank order data is 
often difficult, costly and time-consuming. It is difficult because there is often no current 
method of having customers identify their preferences in an objective manner. Even if 
there were a current way to secure customer's preferences, researchers would require 
statistically significant samples to be reliable, and it would be time-consuming and 
expensive to acquire these samples using traditional methods. 
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SUMMARY OF THE INVENTION 

[0008] Having identified the aforementioned problems in the existing methods of 
consumer surveys and product development and assessment, the inventor has developed 
the present invention. As described in the present application, the invention provides a 
method, system and program product for conducting network-based surveys of 
respondents (such as existing or potential consumers or customers, or other targeted 
respondents) using continuous metric scales and developing a framework within which 
respondents rate new or existing products. A continuous metric scale can be depicted as a 
line, preferably one without end points. When a respondent identifies a rating on the 
continuous line, the rating is recorded as a demarcation on the line. 

[0009] The present invention discloses a method, program product and system for 
conducting product assessment and development, where the method comprises displaying 
on a terminal a predetermined scale against which a respondent can indicate a rating for a 
first existing product, receiving a first existing product rating indicated against the scale 
from the terminal of the respondent, displaying a visual indicator along the scale 
corresponding to the first existing product rating, receiving a first different product rating 
indicated against the scale from the terminal of the respondent, displaying a visual 
indicator along the scale corresponding to the first different product rating and collecting 
ratings made by the respondent via the terminal. 

[0010] The invention also includes a method of conducting assessment and 
development of a product using information from a plurality of targeted respondents (such 
as potential or existing customers or consumers), where the method comprises collecting 
answers and ratings made by a plurality of respondents using the method described above 
and generating a product assessment report for the product. 

[0011] In the present specification, the term "product" is defined broadly to include 
any method, service, or article of manufacture. The term "product" also includes all of the 
features and marketing components of any method, service or article of manufacture, 
including by way of example, but not limited to advertising, marketing plans, brochures, 
packaging, pricing, positioning, and other components that those skilled in the art would 
appreciate. Accordingly, the present invention may be used to assess methods, services, 
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articles of manufacture, and their underlying features and marketing components, whether 
the "product" is pre-existing or new. 

[0012] The invention described herein solves the problem described above regarding 
the use of business plans by providing a reliable and readily replicable objective 
measurement system. The instant invention also solves the problems involved in 
obtaining valid rank-order data by enabling respondents to identify their views on their 
current favorites and their view of the relative appeal of the new product all on the same 
continuous metric line scale. Their current favorites act as a metric anchor. In addition, 
the use of the web-based survey technique enables the developer to obtain statistically 
significant samples on a fast and cost effective basis. 

[0013] Other features and advantages of the present invention will become apparent to 
those skilled in the art from the following detailed description. It should be understood, 
however, that the detailed description and specific examples, while indicating preferred 
embodiments of the present invention, are given by way of illustration and not limitation. 
Many changes and modifications within the scope of the present invention may be made 
without departing from the spirit thereof, and the invention includes all such 
modifications. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0014] The foregoing advantages and features of the invention will become apparent 
upon reference to the following detailed description and the accompanying drawings, of 
which: 

[0015] Figure 1 is a block diagram illustrating a system according to one embodiment 
of the present invention. 

[0016] Figures 2A-2C are flowcharts illustrating the general function and operation of 
an embodiment according to the present invention. 

[0017] Figures 3-15 illustrate various screens displayed to a respondent under a 
preferred embodiment of the present invention. 
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[0018] Figures 16-30 illustrate a report generated under a preferred embodiment of the 
present invention. 

DETAILED DESCRIPTION OF THE INVENTION 

[0019] The present invention is now described in detail in reference to certain 
preferred embodiments of the invention. General features of the invention are first 
described. Next, particular embodiments are described in reference to the above- 
mentioned figures. 

[0020] Generally, the present invention can be summarized as a management 
information system and method for the development and assessment of products and 
services. One preferred embodiment described herein is referred to as a Linescale Product 
Development System. This exemplary system and method relates to a web-based survey 
technique that facilitates developing and testing new products, business investments, and 
value propositions, as well as testing existing products, businesses, and value propositions. 
This technique includes systematic solicitation of respondents, preferably potential 
customers, eliciting data from respondents, automatic processing, interpretation, and 
scoring of the data, and automatic report generation. Using an on-line system, this 
embodiment of the invention provides a fast, engaging, and inexpensive way to gauge 
customer acceptance and use. 

[0021] The interactive development tool first guides generation of the concept, then 
development of the product or service and its marketing components. As explained below, 
the combination of the two techniques, when used iteratively, provides a unique 
management information system, method, and program product for product and service 
development, and business assessment tool. 

[0022] The invention provides a behavior predictive development tool rather than an 
attitude measure. The invention enables respondents, including potential customers, to 
display their preferences for the new concept or product compared to their currently used 
favorites. The invention provides a systematic way to determine how target respondents 
would score a new product concept, how the idea stacks up against the respondent's two 
current favorites, how the execution of the concept or idea being tested performs 
compared to the concept itself and the two favorites, and which competitors or current 
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behaviors they would switch away from to use the new product or service. The system 
processes and translates the data into a readily understandable and replicable score. The 
system provides both quantitative and qualitative development guidance based on scale 
ratings and respondents' verbatim reasons for making their comparative ratings. This 
gives precise guidance to product developers, management, and investors on probable 
trial, repeat and depth of repeat. The analysis shows the user who wants the product and 
why, how to improve it, who hates the product and why, and points to what needs to be 
done to fix the product. 

[0023] The invention relates to a pure development tool built to help companies 
identify, create and build successful new products and fix old products. The iterative 
method of test and repeat allows companies to make bets and see if they pay off - long 
before going to market. The invention recognizes that fast, inexpensive iteration is the 
route to development success and that almost nothing works in first draft. The invention 
provides a system for a company to get the drafts behind them before commitment to 
production and introduction, in a more productive, effective and efficient manner. 

[0024] The invention facilitates the assessment of existing products and businesses on 
a fast and cost-effective basis. It further facilitates product development and business 
investment decisions by providing a quantifiable prediction of customer acceptance of a 
new product. 

[0025] As a product development tool, the invention addresses the issue of how to 
develop products and services on a rapid and cost-effective basis. As the economy 
recovers from the recent slowdown, companies need to replenish their R&D pipeline of 
new products. Because of the past reduction in development spending, there is a gap in 
the traditional development process that requires a rapid screening, development and 
implementation of new products geared to the consumer, professional and small business 
markets. The present invention fills this need by providing a rapid, reliable and 
inexpensive system for obtaining large scale consumer evaluation of new product 
concepts, and developing these concepts into full implementation and execution. 

[0026] This invention relates to a system and method for guiding the concept 
screening and development for a new product or service designed for consumers, 
professionals or small business. After development of the refined concept, the invention 
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guides the development of the product or service itself, including the method of operation, 
customer requirements, product or service performance characteristics and all elements of 
the marketing mix, including brochures, advertising in all media, direct mail offers, 
packaging, premises, etc. The invention allows a user to make better investment decisions 
in new business ventures by providing a quantitative method of determining customer 
acceptance of a product. 

[0027] The invention achieves these results using a web-based survey technique using 
continuous metric scales, translation of the survey results into a readily understandable 
score, and automatic interpretation and reporting of the data and score in plain English. 

[0028] This invention provides the following: a quantitative and objective way of 
predicting acceptance by a group of respondents of a product or service; a step-by-step 
way of eliciting anchored rank order data by using metric anchors; a readily 
understandable score based on a respondent's relative ranking of a product that reliably 
predicts acceptance of the product; an inexpensive way to obtain a large sample of 
respondents, through the web and iteration via the web, thus providing reliability and 
speed at a low cost; a way of improving a concept on a step-by-step basis; a way to 
determine, by the use of norms, when a concept is "good enough," improvable, or whether 
to discard the idea; a way of improving a product or service derived from a concept on a 
step-by-step basis; an automated way to use respondents' relative ranking of items to 
predict acceptance and use over time; a way to predict acceptance by customers or other 
respondents in a way that can be conducted manually, or on a fully or partially automated 
basis, in person or on a remote basis through a communications network; and a way to 
help customers or other respondents accurately report the relative ranking of a new 
product by using continuous metric scales to compare the relative ranking of dissimilar 
products with anchors. 

[0029] One embodiment of the invention facilitates two stages of product 
development: development of the concept by means of a Concept Developer and 
development of the product by the Product Developer. 

[0030] The Concept Developer assesses customers' or other respondents' reaction to 
the concept of the new product or service. To determine customer reaction, the system 
elicits rank order data from the customer. It requests the customer to rank the customer's 
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current favorites, the new concept, and a more detailed description of the concept. Then 
the system asks the customer to rate the idea on a metric scale for believability, 
uniqueness, importance to them personally, and the probability of recommendation to 
friends. 

[0031] In one embodiment of the invention, described in greater detail below, scoring 
is numerical. By way of example, it can be based on six measures, such as the following: 

[0032] a) 0, 5 or 10 on percent preference for concept over current product used and 
next favorite product used - assigning 5 points to preference of concept over each favorite 
(two current products) 

[0033] b) 0 or 10 on percent preference of product experience over concept 

[0034] c) 1 to 6 on believability 

[0035] d) 1 to 6 on uniqueness 

[0036] e) 1 to 6 on importance 

[0037] f) 1 to 10 on will recommend it to friends 

[0038] In this preferred embodiment, calculating the score involves taking a weighted 
average of the answers and ratings in accordance with a predetermined formula. In this 
embodiment, adding up the six numbers, taking the average and multiplying by 100 
derives the score. Preferably, the algorithm to determine the likelihood of customer or 
other respondents' acceptance and trial of the new product is I = ((al+a2)+b+c+d+e+f) / 6 
x 100. Using this algorithm, a system according to the invention displays all of the 
relevant data and scores on one readily understandable metric scale. While one specific 
weighted average is described herein, those skilled in the art will appreciate that any 
weighted average or combination of scores could be used under the present invention. 

[0039] The Product Developer assesses likely customer or other respondent reaction to 
the experience of the new product or service. In the product developer, the system enables 
the respondent to rank the current favorites, the concept, and a more detailed description 
of the concept. Then the system asks the respondent to rate the idea on a line scale for ease 
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of use, perceived speed, charm or pleasantness of experience and probability of 
recommendation to friends. 

[0040] As with the Concept Developer, scoring for the Product Developer of one 
embodiment preferably is numerical, and may be based, for example, on six measures: 

[0041] a) 0, 5 or 10 on percent preference for concept over current product used and 
next favorite product used - assigning 5 points to preference of concept over each favorite 
(two current products) 

[0042] b) 0 or 10 on percent preference of product experience over concept 

[0043] c) 1 to 6 on ease 

[0044] d) 1 to 6 on speed 

[0045] e) 1 to 6 on charming, pleasant 

[0046] f) 1 to 10 on will recommend it to friends 

[0047] Adding up the six numbers, taking the average and multiplying by 100 derives 
the score. For the Product Developer, the preferred algorithm for determining the 
respondent's reaction to the experience of the product compared to his own product and 
the likelihood of adopting the new product is I = ((al+a2) +b+c+d+e+f) / 6 x 100. 

[0048] In the preferred embodiment, the scores for both the Concept Developer and 
the Product Developer provide are then assessed. For example, using the algorithms noted 
above, scores may be ranked as follows: 

[0049] 200 to 300 Forget about it 

[0050] 301 to 400 Not ready for prime time 

[0051] 401 to 500 Something here, but it needs work 

[0052] 501 to 600 Good concept (or other noun), works pretty well 

[0053] 601 to 700 Very strong idea (or other noun), great execution 
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[0054] 701 to 800 Customers are wild about it (or Excellent) 

[0055] In exemplary embodiment of the invention, the key score is the percent of the 
sample scoring the concept or product over 600. In this embodiment, the percentage of 
customers scoring the new concept over 600 would be considered the top score using the 
technique according to the invention. This has a high correlation with subsequent product 
success in the market place. 

[0056] The system generates sample thresholds or norms for product performance 
based on the experience of testing many similar products. The thresholds are the scores 
that successful products achieve. The user of the system may choose a threshold or adopt 
one recommended by the system for the type of product being tested. 

[0057] The system is designed for multiple iterations. Iterations include testing the 
concept or product experience, reviewing the test results, modifying the concept or 
product based on the test results and repeating the tests with new survey participants. 
Both the concept developer and the product developer are used in iterations of the concept 
or product features until the user of the system achieves the desired results. 

[0058] The Concept Developer is iterated until one of two events occurs: either the 
developer gives up, since the concept and a more detailed description of the concept 
cannot be improved to meet target thresholds, or having exceeded target thresholds, 
subsequent iterations do not improve concept performance. 

[0059] The Product Developer works in a similar manner. The system asks the 
customer or other respondent to compare the product concept to various implementations 
of the product, such as on-line applications, advertisements, direct mail offerings, physical 
embodiments of the product, product experiences of taste, wearability, usefulness, quality, 
or many other examples that would be apparent to those skilled in the art. The system 
asks the customer to rank each component under consideration quantitatively and 
objectively to identify how strongly the component supports the concept. Based on the 
customer's ranking, the system generates an overall score and a score of how well each of 
the components supports the product concept. If a key component does not meet 
minimum thresholds, the system generates a plain English message to the user of the 
system. If additional strengthening, addition of elements to the product, or reduction of 
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the price does not improve performance, the system generates important information to the 
developer. 

[0060] The Linescale system is an effective tool for assessing advertising, packaging 
and other marketing components. In another embodiment of the invention referred to as an 
Advertising Developer, the system according to the invention is used to assess the 
persuasive effect of advertisements, packaging or other marketing materials. Using the 
same techniques as the Concept Developer and the Product Developer, the Advertiser 
Developer system asks the customer or other respondent to provide his current assessment 
of the existing or new product and the customer's two other products or relevant 
behaviors. For example, in testing a weight loss product, a customer would be asked to 
assess the weight loss product and his other methods of achieving weight loss, such as 
exercising, taking diet pills or reading healthy cookbooks. After providing relative 
rankings of the product and the customer's two favorite products or behaviors, the 
Advertiser Developer would display a commercial or story board to the customer. After 
viewing the commercial twice, the customer would be asked to re-rate his assessment of 
the product and identify his reasons for the rating both quantitatively and objectively. As 
with the Product Developer and the Concept Developer, the Advertisement Developer 
would generate a detailed report that would provide detailed guidance on the persuasive 
nature of the advertisement. 

[0061] It will be appreciated that the Linescale System according to the invention is 
effective for assessing existing products. 

[0062] Another aspect of the invention described in greater detail below is referred to 
herein as the Marketing Report Card. According to this aspect of the invention, the system 
uses the same techniques as the Concept Developer and the Product Developer to assess 
ongoing products. For example, in testing customer attitudes about a weight loss system, 
the Marketing Report Card asks customers to rate the weight loss system and the 
customer's two other favorite weight loss products or methods. With these competitive 
anchors in place, the system asks the customer to rate each of the components of a system. 
For example, a customer may assess aspects of a weight loss system, such as a diet center, 
staff, food, price, and instructional materials. After presenting the customer with a score 
for his ratings, the system asks the customer whether he agrees or disagrees with the 
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assessment he has provided and the system asks the customer for quantitative and 
subjective reasons for the rating. The system generates a report that provides valuable 
feedback about what aspects of a product work and which do not in a rapid and easily 
understandable manner. 

[0063] It will be appreciated that one aspect of the present invention that creates the 
reliability and stability of the data generated by the Linescale System is the way it creates 
the "competitive frame" or "customer space." This means that the system helps place the 
respondent in the right frame of mind to answer the questions in a reliable fashion. 
Without establishing this framework, customers may give answers that do not provide an 
accurate predictor of their behavior. Without the framework, respondent answers may be 
influenced by whim or a typical human response to novelty, order of questions, or the 
difficulty of identifying true preference in the abstract. 

[0064] The personalized selection of favorite brands, products or practices in the 
market space as competitive frame is a unique stabilizing factor as well as a benchmark for 
evaluation. Market research today typically may use a "control" such as a given brand or 
product currently offered in the market place as an "anchor". Another common practice is 
to use an "ideal" imagined product as a control. But each of these lacks the personal 
commitment and clarity of understanding based on the respondent's personally selected 
items or practices as current favorites. 

[0065] The method for establishing the framework is to ask the customer or other 
respondent to identify his or her current favorite activity or brand and then to identify his 
or her second favorite activity or brand. The system often provides the respondent with a 
wide array of activities and brands as choices to get the customer in the right space. 
Products and services may cross many markets, so it is often desirable to identify the 
various activities associated with the product to give the respondent the right framework. 

[0066] It will be understood by those skilled in the art that the invention may be 
deployed in many different embodiments. The following describes preferred 
embodiments of the invention in reference to the accompanying drawings. 
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[0067] Figure 1 is a block diagram showing a system according to one preferred 
embodiment of the system. While this exemplary system is web-based, it will be 
appreciated that other approaches may be utilized. 

[0068] The system 1 includes a host computer 10 such as a server computer, a network 
1 1, a user terminal 12 and an array of terminals, generally referred to by references 13-15. 
The terminals may comprise personal computers, or other types of terminals known in the 
art. Target respondents (typically comprising, for example, prospective or existing 
consumers or customers) of the subject product, use the terminals 13-15 to provide 
feedback about a product concept to the host computer 10. In the embodiment of the 
system discussed herein, a survey preferably is conducted with a minimum of 48 
respondents, but other numbers of respondents could be used. Thus, it should be 
understood that while four terminals illustrated, a larger number will preferably be used. 

[0069] As illustrated in the examples that follow, the host computer 10 transmits 
screens to the terminals 13-15 over the network 11. The host computer 10 also transmits 
reports to the user of the Linescale system at the user's terminal 12 over the network 11. 
The user may also use the user terminal 12 to access the host computer 10 to obtain the 
reports from the surveys, as described generally below. 

[0070] Figures 2A-2C are flow charts showing general features of operation of a 
system according to one embodiment of the invention, as described in more detail below. 
It will be understood that steps illustrated could be repeated; further, the steps would not 
necessarily be follow in the specific order illustrated. 

[0071] Step S10 represents a step of contacting respondents that may comprise, for 
example, potential or existing customers, consumers or other targeted groups of 
respondents, and soliciting their participation. This may be accomplished, for example, by 
a letter, electronic message or other communication that includes a URL to access a 
system that conducts the survey. Once the respondent logs onto the system, the 
respondent receives and provides information that is used for concept and product 
development and to generate reports. 

[0072] In the example shown, a respondent begins a survey at S20. Preferably, the 
respondent provides information about the respondent's current practices and/or 
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preferences with "respect to an existing product, which as noted above, may comprise any 
method, service or article of manufacture, as well as all of the features and marketing 
components of any method, service or article of manufacture. Based on these responses, 
the system displays a summary of the respondent's current experiences at S30. 

[0073] The system then prompts the respondent and receives ratings for the existing 
product at S40. According to one feature of the invention, this is done in reference to a 
metric scale or "linescale." The system displays a visual indicator along the scale 
corresponding to the first existing product rating as represented at S50. This may be 
repeated for several different aspects of the existing product. 

[0074] At S60 of this example, the system displays a "header concept" or general 
introduction of a new product. While reference is made to a new product, it will be 
understood that the new product could simply be different from the first product. The 
system receives from the respondent and displays ratings of the new product on the 
predetermined scale used for the existing product at S70. As generally illustrated, this is 
repeated for several different aspects of the new product 

[0075] Once the ratings of the new product are displayed, the system preferably 
displays a detailed description of the new product at S80. The respondent then is 
prompted to again rate the new product at S90 and a visual indicator of the rating of the 
new product is displayed at SI 00. Preferably, this is performed successively for various 
aspects of the new product. 

[0076] Once the ratings are collected, at SI 10 the system generates a score, preferably 
by taking the average of the total responses and multiplying them by, for example, 100. 
The system then displays the score and values to the respondent at SI 20. Preferably, the 
respondent is prompted to validate the scores at SI 30, for example, by indicating whether 
the respondent agrees or disagrees with the scores. Preferably, the respondent can also 
provide reasons for the ratings and other comments. 

[0077] According to another aspect of the invention, illustrated generally at SI 40, the 
system collects and processes the scores from several respondents to generate a report. 
Aspects of the report may include those described more specifically below in reference to 
one exemplary embodiment. 
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[0078] An example of one embodiment of the invention is now described in reference 
to a bill handling product. As generally indicated above, respondents are first contacted 
and requested to participate in a survey. For example, the system may generate 
solicitation notices to potential customers in the target market by electronic mail. For 
example, the message indicates that the request is for surveys used for future product 
ideas. Preferably, it includes an incentive for participation, such as entry in a prize 
drawing. Preferably, the message provides an overview of the survey, for example, 
indicating that it will: ask the respondent to indicate what he or she does today on a 
continuous metric scale; show new ideas and ask how to rate them on the same continuous 
metric scale; recap the rating and what it means and ask the respondent whether he/she 
agrees or disagrees. Preferably, included in the message is the URL for access to the 
survey. 

[0079] Once the respondent begins the survey, the first screen asks the respondent 
questions related to his or her current bill payment practices. For example, it may ask: 
"Are you the person who pays the bills in your household?"; "About how many bills does 
your household pay per month?"; "How do you usually handle your bills?"; and "You 
sometimes also...". As illustrated in Figure 3, the respondent responds by selecting from 
answers provided on a drop down menu or in a text box for other responses. In the 
example shown in Figure 3, the respondent has responded that he is the bill payer, he pays 
nine bills a month, and pays most by check, but a few electronically. 

[0080] According to one feature of the invention, the system then displays a 
predetermined scale against which the respondent can indicate a rating for an existing 
product. Preferably, the scale is based on the information previously furnished by the 
respondent in the preceding screens (such as illustrated in Figure 3). A series of such 
continuous metric scales are shown in Figures. 4A-4C. For example, Figure 4 A, illustrates 
a screen which states the respondent's current practice of bills coming in the mail and 
paying most by check, and prompts the respondent to rate his or her experience of paying 
bills by check on a metric scale that runs from excellent to terrible. Typically, the 
respondent may indicate his or her preference using a mouse to select a point on the scale 
and then confirm the result. While a mouse is described above as a user input device for 
indicating ratings on a continuous metric scale, it would be apparent to one skilled in the 
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art that any number of other electronic input devices could also be used in the present 
invention. 

[0081] The system then validates what the respondent has said by marking the 
continuous metric scale with the respondent's response to the question of how the 
respondent rates his or her current bill payment experience of paying bills by check. In 
this example, the respondent rated the experience as "poor." On the same screen, the 
system asks the respondent to rate his on-line bill payment experience on the same 
continuous metric scale from excellent to terrible. 

[0082] Next, as shown in Figure 4C, the second rating for paying on-line is displayed 
simultaneously with the first rating, the system displays to the respondent both of his 
responses on one continuous metric scale. In this example, the respondent reported that 
his experience of paying on-line is good and that his experience of paying by check is 
poor. In addition, the system prompts the respondent to "Click here to see the concept." 

[0083] The system next displays to the respondent a "headline" concept statement 
about the new product. An exemplary headline concept is as follows: 

Secure and Organized Bill Handling 

Finally, an easy way to take the drudgery out of getting your bills paid on time. With this 
new service you can get your utility bills, credit card bills, telephone, Internet ISP, Cable, 
student loan bills, insurance - even mortgage or rent - right on the web! You'll get an 
email as soon as any bill or invoice is ready for viewing. Then you go to your private web 
site located behind your bank's security firewall. Since you can set up reminders and 
alerts to notify you anyway you like (email, instant messenger, even a text message sent 
your cell phone!), you won't have to worry about missing a payment or a due date ever 
again. Although you can access all you relevant bills and documents in one easy place, 
each bill is coming directly from your biller. No one else, including your bank, can see or 
access the information. 

You simply log on to your personalized web site and you can: 

• View all the details of your bills and statements 

• Pay bills electronically right from any bank account or charge them to a credit card 

• Store these bills and documents online for future reference 

• Print bills and statements for your paper files 

• View a complete organized payment history and archive 

• Share a specific bill or statement with someone else - for example, share the 
tuition bill with your parents 

This service will present and archive the following bills and statements to make your life a 
little more manageable. 



002.1183616.1 



-16- 



Attorney Docket No. 036167-0104 



• Utility bills (phone, electric, cable, etc.) 

• Rent or mortgage 

• Car Loan bills 

• Credit Card bills 

• Insurance bills 

• Medical bills 

• Tuition bills 

• School Housing, Meal Plan, Student Health, Bookstore bills 

• And practically any other bill from any business or institution 

[0084] After the respondent reviews the concept, a next screen (see Figure 5) prompts 
the respondent to rate the concept on a continuous metric scale that reflects the 
respondent's previous rating of his current bill paying experience. The respondent's 
ratings of his current experience are called "metric anchors." This innovative feature 
helps get valid and replicable data from the respondent. By recording the respondent's 
view of the respondent's current experience, the respondent articulates his or her baseline 
experience on an objective basis. 

[0085] Once the respondent marks his rating, the system displays a screen that reflects 
the respondent's rating, as shown, for example, in Figure 6. In this example, the 
respondent rated the New Concept as very good. On the same screen, the system asks the 
respondent to "Click here to see how it works." 

[0086] Next, the respondent views a more detailed description of how the new product 
or concept works as shown in Figures 7-11. Preferably, the system describes the product 
in a detailed narrative, for example, as illustrated in Figure 7. Preferably, the system 
subsequently displays a series of interactive screens, which provide tasks to the respondent 
that give the respondent a typical experience with the new product. 

[0087] In this example, interactive screens relate to a bill payment system for a bank. 
It includes a series of actions that can be performed for a checking account, including a 
summary of accounts, detail of accounts, account summary, account download, stop 
payment, order checks, transfer funds, and pay bills. The screens include a drop down 
menu to select from existing accounts. The available balance is also displayed. 
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[0088] As shown in Figure 8, a personalized welcome screen displays a series of new 
bills, including the biller, the date received and their status. The screen includes option 
buttons to add a biller, delete a biller, and edit preferences. 

[0089] Figure 9 illustrates a screen for selecting the method of payment. Figure 10 
illustrates a screen for viewing a bill. Figure 1 1 is a partial view of an account summary 
page. It will be understood that the illustrated screens are simply examples of what may 
be displayed to permit the respondent to experience a new product. 

[0090] At the end of the experience, in the example of Figure 1 1 , the system prompts 
the respondent to "Click here to continue." 

[0091] On the next screen, as shown in Figure 12A, the system prompts the 
respondent, "Now that you know more about the idea, how do you rate it - higher or 
lower?" The screen prompts the respondent to place the respondent's rating of the more 
detailed understanding of the concept and the concept experience on the same scale as the 
respondent's rating of the concept and the respondent's two current favorites. 

[0092] Once the respondent inputs a rating, the system, as shown in Figure 12B, tells 
the respondent how the respondent rates the new concept overall. In this embodiment, the 
respondent rates the New Experience as Very Good and better than his rating of the New 
Concept and his current bill payment practices of paying on-line and paying by mailing a 
check. In the same screen, the system tells the respondent to "click here" for a few more 
questions. 

[0093] The system then asks the respondent to rate the new idea on a series of 
continuous metric scales. For example, a series of four lines scales may be provided to 
allow the respondent to successively indicate: ease of use, speed, charm, and whether the 
respondent would recommend the new product to friends. It does this by asking the 
respondent to answer the following questions: "How easy or hard is it to use?", "How fast 
or slow is it?", "Is it charming and pleasant or ugly?", and "How likely are you to 
recommend it to friends?" For each answer, the system generates a score and displays the 
answer and the score to the respondent. The final screen showing all four answers is 
shown illustrated at Figure 13. This feature ensures that the respondent understands what 
his answer is and the score. 
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[0094] Although a last screen of this sequence is shown in Figure 13, preferably, the 
respondent is prompted on a step-by-step basis. For example, first the system asks, 
"Based on what you know so far about the idea and how it works, how would you rate it 
for ease of use? Click on the line from 'Easiest' to 'Terribly Hard' to tell us what you 
think." In this example, the respondent rates the idea as Very Easy, and the system 
generates a score of 41.4. The system then responds with a screen confirming the 
respondent rates the idea as Very Easy and tells the respondent that the score is 41.4. On 
this same screen, the system asks the respondent the next question, "Based on what you 
know so far about the idea and how it works, how would you rate it for speed? Click on 
the line from 'Fastest' to 'Painfully Slow' to tell us what you think." In this example, the 
respondent rates the speed as "Very Fast," the system generates a score of 44.4 and 
displays a confirmation screen depicting the respondent's rating and score. In the same 
screen, the system may ask the next question, "How pleasant was the idea? Click on the 
line from 'Charming' to 'Ugly' to tell us what you think." 

[0095] In this example, the respondent rates how pleasant the idea was as "Very 
Pleasant." The system generates a score of 44.7 and displays a confirmation screen, which 
shows the respondent's rating and score. On this screen, the system asks the final 
question, "Would you tell your friends about the idea? Click on the line from 'Yes' to 'No 
way' to let us know." 

[0096] In this example, the respondent marks it as, "Yes, I would absolutely tell my 
friends about this and ask them to try it." The system generates a score of 41.2, and 
displays a confirmation screen, such as Figure 13, which shows the respondent's rating 
and score. It asks the respondent to "Click here to finish." 

[0097] In the preferred embodiment, the system transmits respondent's responses over 
the network to the host computer. Once information corresponding to the respondent's 
responses is received by the host computer, it calculates the score by recording a number 
for each response on the continuous metric scale to questions about the new product 
concept and more detailed explanation. The system computes the score by taking the 
average of the total responses and multiplying by 100. 

[0098] According to another feature of the invention, the system then provides the 
respondent a report, such as in the form shown in Figure 14. In this example, the report 
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indicates to the respondent that the respondent's personal rating for the concept was 750, 
out of a scale from 200 to 800. 



[0099] Preferably, the system tells the respondent what the score means in easy to 
understand language. In this example, it says: 

[00100] "Here are the scores and their values: 

[00101] 200 to 300 Forget about it 

[001 02] 30 1 to 400 Not ready for prime time 

[00103] 401 to 500 Something here, but it needs work 

[00104] 501 to 600 Good concept, works pretty well 

[00105] 601 to 700 Very strong idea, great execution 

[00106] 701 to 800 I'm wild for it 

[00107] In this example, the system asks the respondent whether he agrees with his 

score, and whether he would rate it higher or lower. The system then asks the customer 



for the reasons for his rating to be typed in a text box. Then the respondent is asked to 
select a number of reasons from a checkbox list of possible reasons for his rating. 

[00108] As shown in Figure 15, the respondent is then prompted to provide certain 
information to help assess the respondent. As illustrated, these may include demographic 
and relevant behavior questions, as well as questions to gauge the attitude of the 
respondent. 

[00109] From the foregoing, it will be appreciated that the "Linescale" technique can be 
used advantageously to evaluate both concepts and products. It will further be appreciated 
that various ratings can be utilized on the scale, including descriptive ratings as well as 
numerical ratings. Further, different numerical values or weights may be attached to 
particular rating points on a scale. Preferably, the ratings are assigned a corresponding 
score that can be used to provide an overall evaluation. Preferably, the respondent views 
each continuous metric scale successively so that a respondent can provide ratings for a 
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first, existing product or concept, and then rate a new or different product or concept 
against the same scale. 

[00110] Another aspect of the invention is the detailed and automatic report that the 
system generates for the product developer or user of the system. This report is referred to 
herein as a product assessment report. The report preferably can be furnished using a 
web-based system as illustrated generally in Figure 1, wherein the host computer 10 of the 
system transmits the report to the terminal 12 of the product developer or user of the 
system over the network 1 1 . Preferably, the host computer may also notify the user by e- 
mail that the report is ready, and the user accesses the report from the host computer over 
the network. Other implementations for delivery of the report will, of course, be apparent 
to those skilled in the art. 

[00111] Preferably, data is reported automatically in a decision rule-driven reporting 
format. Preferably, reporting is real time. As each respondent answers, his responses are 
added to the database, and new calculations are automatically generated. There are both 
individual concept and product reports, and an aggregate "Scorecard" which displays a 
number of concepts or products and their scores. 

[00112] An example of such a report is shown in Figures 16-30. In this report, the 
system automatically does the following: summarizes the new product concept that was 
shown to the target customers or other respondents who took the survey (see Figures 16 
and 17); reports the scores in the correct tables, and the reasons for the score, (see Figures 
16-30, wherein scores are referred to as "I-CAT scores), and generates table headings and 
table heading narratives in plain English acknowledging sample size, preferences, 
percentages, scores, etc., based on decision rules. 

[00113] As illustrated, the system interprets the data and analysis and reports this 
interpretation in plain English, based on decision rules. For example, if the preference for 
the experience over the concept is higher than a predetermined percentage, the analysis 
states, "The product delivers on the promise." Examples of other automatically generated 
messages, based on predetermined percentages include, by way of example, "The product 
supports the concept, but needs improvement," or "The product does not deliver on the 
promise." 
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[00114] The system also creates a unique scorecard generator, which allows for the 
custom creation of a scorecard with comparisons to scores of any selected concepts or 
products. It further allows the user to change descriptive names of the selected concepts 
for confidentiality or other purposes. The system allows or denies linkage to the 
individual report supporting the score. The system provides the data in rank order 
determined by the top score. The system highlights key information, such as the Score 
Segments and reports the size of the market segment among which the test was conducted. 
The system also allows the user to edit the reports. 

[00115] With this report, the product developer has the information needed to decide 
whether to modify the product, keep it as is, or discard it. 

[00116] As noted above, embodiments within the scope of the present invention include 
program products comprising computer-readable media for carrying or having computer- 
executable instructions or data structures stored thereon. Such computer-readable media 
can be any available media that can be accessed by a general purpose or special purpose 
computer. By way of example, such computer-readable media can comprise RAM, ROM, 
EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or 
other magnetic storage devices, or any other medium which can be used to carry or store 
desired program code in the form of computer-executable instructions or data structures 
and which can be accessed by a general purpose or special purpose computer. When 
information is transferred or provided over a network or another communications 
connection (either hardwired, wireless, or a combination of hardwired or wireless) to a 
computer, the computer properly views the connection as a computer-readable medium. 
Thus, any such connection is properly termed a computer-readable medium. 
Combinations of the above are also to be included within the scope of computer-readable 
media. Computer-executable instructions comprise, for example, instructions and data 
which cause a general purpose computer, special purpose computer, or special purpose 
processing device to perform a certain function or group of functions. 

[00117] The invention is described in the general context of method steps, which may 
be implemented in one embodiment by a program product including computer-executable 
instructions, such as program code, executed by computers in networked environments. 
Generally, program modules include routines, programs, objects, components, data 
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structures, etc. that perform particular tasks or implement particular abstract data types. 
Computer-executable instructions, associated data structures, and program modules 
represent examples of program code for executing steps of the methods disclosed herein. 
The particular sequence of such executable instructions or associated data structures 
represents examples of corresponding acts for implementing the functions described in 
such steps. 

[001181 The present invention in some embodiments, may be operated in a networked 
environment using logical connections to one or more remote computers having 
processors. Logical connections may include a local area network (LAN) and a wide area 
network (WAN) that are presented here by way of example and not limitation. Such 
networking environments are commonplace in office-wide or enterprise-wide computer 
networks, intranets and the Internet. Those skilled in the art will appreciate that such 
network computing environments will typically encompass many types of computer 
system configurations, including personal computers, hand-held devices, multi-processor 
systems, microprocessor-based or programmable consumer electronics, network PCs, 
minicomputers, mainframe computers, and the like. The invention may also be practiced 
in distributed computing environments where tasks are performed by local and remote 
processing devices that are linked (either by hardwired links, wireless links, or by a 
combination of hardwired or wireless links) through a communications network. In a 
distributed computing environment, program modules may be located in both local and 
remote memory storage devices. 

[00119] An exemplary system for implementing the overall system or portions of the 
invention might include a general purpose computing device in the form of a conventional 
computer, including a processing unit, a system memory, and a system bus that couples 
various system components including the system memory to the processing unit. The 
system memory may include read only memory (ROM) and random access memory 
(RAM). The computer may also include a magnetic hard disk drive for reading from and 
writing to a magnetic hard disk, a magnetic disk drive for reading from or writing to a 
removable magnetic disk, and an optical disk drive for reading from or writing to 
removable optical disk such as a CD-ROM or other optical media. The drives and their 
associated computer-readable media provide nonvolatile storage of computer-executable 
instructions, data structures, program modules and other data for the computer. 
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[00120] Software and web implementations of the present invention could be - 
accomplished with standard programming techniques with rule based logic and other logic 
to accomplish the various database searching steps, correlation steps, comparison steps 
and decision steps. It should also be noted that the word "component" as used herein and 
in the claims is intended to encompass implementations using one or more lines of 
software code, and/or hardware implementations, and/or equipment for receiving manual 
inputs. 

[00121] The foregoing description of embodiments of the invention has been presented 
for purposes of illustration and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed, and modifications and variations are possible 
in light of the above teachings or may be acquired from practice of the invention. The 
embodiments were chosen and described in order to explain the principals of the invention 
and its practical application to enable one skilled in the art to utilize the invention in 
various embodiments and with various modifications as are suited to the particular use 
contemplated. 
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